Artificial Intelligence & Medical Research: What Works...and What Does Not
Artificial Intelligence (AI) can be utilized to help your medical research, but some things work and others do not. This brief guide will help you understand the current state of the effectiveness of using AI as of July 2024.
For these examples I am using the basic version of ChatGPT; there would presumably be differences with the Pro version or other tools.
One thing I am noticing is that AI has trouble understanding questions, even ones that seem direct. For example, when asking for help reviewing the syntax of my search strings, ChatGPT just reformats it. When I again ask to point out any errors I have made, it simply reformats the search. I recently did a search where the issue was quote marks around search terms; ChatGPT could not point out what was wrong. (This is also caused by the chat box reformatting the font–something a human would need to be aware of.) Even in the above image, it took multiple attempts for Dalle 3 in Bing to produce something close to what I was asking for.
Like with search keywords, AI’s “understanding” can be a bit too literal. When you use the research databases, it is ideal to both review exactly how the database is using your keywords and to also employ subject terms, like MeSH. The human ability to both index and review literature is still important. AI struggles in a similar vein. When I ask AI to give me an image of Wally (“Waldo” for Americans or وين والي؟/وين فضولي؟ in Arabic) from Where’s Wally? (“An image of Where's Waldo as found in the Waldo books.”) The resulting image shows both that it knows who Wally is, but also that it does not really understand the concept of “Where’s Wally?”
Be aware that AI is also evolving and changing. One important thing with research is documentation. If you use AI to help searches, it is advised to note your specific questions, the answers you receive, and on what platform and when you asked your questions. Even with that, it is (like Google Scholar to an extent) a “black box” wherein the details and mechanisms of your search/questions remain unclear and perhaps un-reproducible.
Note: some AI tools offer only a limited number of searches during a certain time period. I ran into that limitation with ChatGPT, where is locked me out for a few hours before letting me continue my searches.